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The  University  of  Georgia 

Franklin  College  of  Arts  and  Sciences 
D^artment  of  Psychology 

March  8,  1993 


Scientific  Officer  Code:  1142CS 
Susan  E.  Chlpman 
Office  of  Naval  Research 
800  North  Quincy  Street 
Arlington,  VA  22217-5000 

Dear  Susan: 


This  quarterly  report  describes  our  progress  toward  the  goal  of  separating 
the  Influence  of  subject  matter  knowledge  from  that  of  reading  skill  on 
performance  in  a  reading  comprehension  test.  At  this  time,  we  are  focusing  on 
better  ways  to  measure  the  quality  of  subject  matter  knowledge. 
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Two  aspects  of  the  quality  of  subject  matter  knowledge  are  (a)  its 
correspondence  to  the  knowledge  of  qualified  experts,  and  (b)  its  internal 
consistency,  sometimes  called  goodness  (McClelland  St  Rumelhart,  1988),  energy 
(Hopfield,  1984),  harmony  (Smolensky,  1986),  or  "harmany”  (Thagard,  1989).  Both 
aspects  are  measured  by  the  cognitive  structure  test. 

During  the  current  grant,  we  have  just  completed  a  computer  program  that 
takes  as  input  a  list  of  terms  provided  by  an  expert  (or  from  a  file),  and  then 
immediately  constructs  and  administers  a  cognitive  structure  test  based  on  it, 
and  provides  plots  of  the  results  on  the  screen  and  printer,  including  linear 
plots,  networks,  and  multidimensional  scaling  results.  These  are  Intended  to 
make  for  productive  interviews  with  experts  to  determine  the  meaning  of  the 
knowledge  structure  they  have  produced,  which  greatly  helps  with 
interpretability,  as  well  as  helping  to  specify  alternative  views  thought 
legitimate  by  the  experts.  This  program,  which  we  call  CSTST,  standing  for 
Cognitive  STructureS  Test,  is  available  for  your  use.  It  is  a  research  tool,  so 
it's  easy  to  use  only  once  you  know  how,  and  I  will  be  glad  to  show  you  how. 

We  were  already  well  advanced  in  our  methods  for  measuring  the  novices' 
correspondence  to  the  experts'  knowledge  structure,  using  quantitative  methods 
Including  correlation,  principal  components  analysis,  network  analysis,  and 
nultidimensional  scaling.  The  other  main  aspect  of  the  quality  of  subject  matter 
cnowledge  that  we  are  investigating  is  its  goodness  (or  energy,  harmony,  or 
harmany).  The  aspect  of  quality  we  have  measured  is  something  like  the  non¬ 
technical  mewing  expressed  by  the  term  "internal  consistency"  (be  careful  here 
not  to  use  the  meaning  of  this  term  used  in  psychological  testing).  This  refers 
to  whether  the  different  parts  of  the  person's  subject-matter  knowledge  "fit 
together"  well.  We  measure  this  by  measvurlng  the  extent  to  which  the  person's 
associative  network  "bears  thinking  about."  The  person's  associative  network  of 
sxibject  matter  knowledge  is  composed  of  certain  constraints  wl  Ich  hold  between 
the  elements  of  his  knowledge;  we  get  these  constraints  when  we  give  the 
cognitive  structure  test.  We  then  measure  the  degree  to  which  the  constraints 
are  satisfied  by  the  thinking  process  of  spreading  activation,  which  we  simulate 
on  the  computer.  ^ 
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We  have  now  completed  analysis  o£  4  experiments  with  a  total  of  27  experts 
and  652  novices  In  which  we  measured  the  quality  of  the  subjects'  prior  knowledge 
In  this  way,  and  all  confirmed  our  hypotheses.  I  enclose  the  paper.  I  have 
marked  the  parts  where  the  relevance  to  the  present  project  Is  pointed  out  most 
explicitly. 

Wallace  Slnalko  has  asked  me  to  keep  him  informed  about  the  progress  of 
this  project,  so  I  have  sent  him  a  copy  of  this  letter  and  the  paper.  I  have 
also  sent  them  to  the  other  people  on  the  distribution  list. 

I'm  aware  that  I  have  not  always  been  as  clear  In  these  reports  as  I  would 
wish.  If  euiybody  has  any  questions,  please  call  me  at  (706)  542-3094,  or  at  home 
at  (706)  549-4661,  or  e-mail  at  IBRSUGA.cc.uga.edu.  or  fax  at  (706)  542-6064  or 
3275.  As  always,  I  look  forward  to  hearing  from  you. 


Sincerely, 


Biruce  K.  Britton 
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ABSTRACT 

This  paper  reports  four  experiments  showing  that  27  experts'  mental  representations  had 
significantly  higher  harmony  and  faster  settling  rates  than  652  novices'  when  activation  was 
spread  through  the  representations  In  a  simulation  of  thinking;  that  when  coherent  texts  were  read 
by  novices,  they  produced  mental  representations  with  significantly  higher  harmony  and  faster 
settling  rates  than  less  coherent  texts;  and  that  novices  whose  representations  matched  the 
experts'  mental  representations  had  significantly  higher  harmony  and  faster  settling  rates.  The 
results  were  found  for  declarative  experts  In  history  and  procedural  experts  In  literary 
Interpretation,  for  Dutch  and  American  experts,  for  novice  groups  Including  U.S.  Air  Force  recruits 
and  U.S.  and  Dutch  undergraduates,  and  for  both  history  texts  and  literary  texts.  These  results 
were  consistent  with  our  hypothesis  that  the  Quality  of  a  person's  prior  knowledge  determtnesThe 
^armony  and  secillng  rates  of  their  representations  and  tiiaiYhese  can  be  measured  by  simulating 
^.Jhejspread  of  activation  through  the  person's  mental  representation  of  a  subject  matter  domain 
Harmony  may  also  be  used  as  a  metacognitive  signal. 
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ABSTRACT 

This  paper  reports  four  experiments  showing  that  27  experts'  mentai  representations  had 
significantly  higher  harmony  and  faster  settling  rates  than  652  novices'  when  activation  was 
spread  through  the  representations  in  a  simulation  of  thinking;  that  when  coherent  texts  were  read 
by  novices,  they  produced  mental  representations  with  significantly  higher  harmony  and  faster 
settling  rates  than  less  coherent  texts;  and  that  novices  whose  representations  matched  the 
experts'  mental  representations  had  significantly  higher  harmony  and  faster  settling  rates.  The 
results  were  found  for  declarative  experts  in  history  and  procedural  experts  In  literary 
Interpretation,  for  Dutch  and  American  experts,  for  novice  groups  Including  U.S.  Air  Force  recruits 
and  U.S.  and  Dutch  undergraduates,  and  for  both  history  texts  and  literary  texts.  These  results 
were  consistent  with  our  hypothesis  that  the  quality  of  a  person's  prior  knowledge  determines  the 
harmony  and  settling  rates  of  their  representations  and  that  these  can  be  measured  by  simulating 
the  spread  of  activation  through  the  person's  mental  representation  of  a  subject  matter  domain. 
Harmony  may  also  be  used  as  a  metacognitive  signal. 

in  these  studies,  we  Investigated  the  quality  of  mentai  representations.  To  do  this,  first  we 
measured  each  subjects  mental  representation  for  a  domain,  and  then  we  simulated  the  subjects 
thinking  process  about  it  We  then  compared  the  end  product  of  the  thinking  process  with  the 
original  mental  representation;  a  good  match  indicated  a  high  quality  representation,  in  the  sense 
that  the  constraints  in  the  original  representation  had  been  satisfied.  Another  index  that  we  used 
for  the  quality  of  the  representation  was  the  number  of  processing  cycles  required  for  the 
simulated  thinking  process  to  settle. 

We  measured  the  mental  representations  of  experts  and  novices  by  asking  them  to  rate 
the  relatedness  of  all  possible  pairs  of  12  terms  from  a  text  The  result  was  a  fully  inter-connected 
network  of  associations.  To  simulate  the  thinking  process,  we  spread  activation  through  this  fully 
interconnected  network  of  associations  until  it  settled  into  a  stable  state.  Besides  observing  the 
time  it  took  to  settle  (here  called  settling  rate)  we  also  calculated  a  harmony  score,  which  is  a 
measure  of  the  ‘goodness*  of  the  mentai  representation,  based  upon  the  degree  to  which  the 
settled  representation  satisfies  the  constraints  Inherent  In  the  original  ratings  of  relatedness 
(McClelland  &  Rumelhart,  1986).  Our  hypothesis  was  that  subjects  with  better  prior  knoyyjedge  . 
would  produce  mental  representations  which  better  satisfied  the  constraints,  leading  to  higher 
"^harmony  TCores  and  faster  settling  rates. 

Tiie'ex^entia’cduriterpaftTo  oilr  Vimulatlon  would  be  a  person  thinking  about  a  subject 
matter  area  that  they  knew  something  about  After  completing  their  thinking  process,  the  person 
would  compare  the  end  product  of  it  to  the  constraints  Inherent  In  the  network  of  ideas  s/he 
started  with.  If  they  matched  perfectly,  then  the  person  would  conclude  that  s/he  had  a  ‘good* 
representation,  corresponding  to  a  harmony  score  of  1.0.  But  if  they  did  not  match  perfectly,  then 
the  representation  would  be  judged  less  good,  with  a  harmon^  score  below  1 .0;  harmony  can  go 
as  low  as  -1.0.  The  person  could  use  the  feeling  of  ‘goodness*  reflected  In  the  harrnony  score  as 
an  Index  of  how  well  s/he  understood  the  subj'ect-n^ite^f'an^ea,  or  fdfofher  purposes. 

In  our  simulation,  the  person's  knowl^ge  about  the  subject  matter  area  was  represented 
by  an  associative  network  in  the  computer  program,  with  the  network's  links,  and  the  strengths  of 
those  links,  constructed  from  the  person's  ratings  of  relatedness  of  the  12  terms.  The  person's 
thinking  about  this  network  of  association  was  Emulated  by  spreading  activation  along  the  links  in 
accord  with  their  strengths  until  the  network  settled. 

What  are  Harmony  and  Settling  Rate? 

Some  simple  illustrations  of  the  spreading  activation  process  will  clarify  the  notions  of 
harmony  and  settling  rate.  Consider  first  the  two  simplest  possible  networks,  those  of  r.vo  nodes. 


A  and  B,  connected  by  a  single  link,  which  can  be  either  excitatory,  represented  by  a  + 1 ,  or 
Inhibitory,  represented  by  a  -1,  shown  In  Figure  1.  An  excitatory  link  is  equivalent  to  a  positive 
association,  corresponding  to  the  notion  that  thinking  of  one  idea  leads  to  thinking  of  another  idea: 
If  one  thinks  of  A,  one  will  think  of  B,  and  vice  versa.  An  inhibitory  link  is  a  negative  association, 
corresponding  to  the  notion  that  thinking  an  idea  leads  to  not  thinking  of  another  idea;  If  one 
thinks  of  A,  one  will  be  less  likely  to  think  of  B,  and  vice  versa.  If  we  start  to  spread  activation  from 
A  to  B  and  B  to  A,  and  allow  the  network  to  settle  into  a  stable  state,  both  of  these  networks  will 
settle  very  quickly,  and  their  final  state  will  satisfy  the  constraints  perfectly,  leading  to  a  harmony 
score  of  + 1 .0.  That  is,  for  the  excitatory  network  both  nodes  will  be  on  after  the  network  settles, 
entirely  consistent  with  the  constraint  that  both  nodes  should  be  in  the  same  state,  expressed  by 
the  + 1 .  So  the  harmony  score  will  be  at  its  maximum  of  -i- 1  because  the  final  state  of  the  network 
Is  completely  consistent  with  the  constraints. 


Figure  1 

Two-Node  Networks 


For  the  Inhibitory  network,  one  node  will  be  on  and  the  other  will  be  off,  consistent  with  the 
constraint  that  the  nodes  should  be  in  different  states,  expressed  by  the  -1.  Again  the  harmony 
score  will  be  at  Its  maximum  of  + 1  because  tne  final  state  matches  the  constraints  perfectly. 

Harmony  scores  of  less  than  one  can  occur  in  three-node  networks.  For  example, 
consider  the  network  at  the  left  of  Figure  2.  Here  A  Is  excitatorially  linked  to  B  and  C,  but  B  and  C 
are  Inhibitoriatly  linked  to  each  other.  The  thought  that  is  equivalent  to  this  network  can  be 
expressed  In  words  as  Two  things  that  are  different  from  each  other  (I.e.,  B  and  C)  can  both  be  the 
same  as  something  else  (I.e.,  A)."  In  trying  to  simulate  the  thinking  of  this  thought,  the  spreading 
activation  process  may  start  out  with  A  in  some  state,  say  S.  Then  B  will  tend  to  be  in  the  same 
stats  S,  because  of  the  excitatory  link  between  A  and  B  represented  by  the  -i- 1 .  Then  C  will  tend  to 
be  In  a  different  state,  say  Not-S,  because  of  the  inhibitory  link  between  B  and  C.  But  if  C  Is  In  the 
state  Not-S,  then  A  will  tend  to  be  In  the  same  state,  namely  Not-S,  because  of  the  excitatory  link 
between  C  and  A  represented  by  the  1 .  But  if  A  Is  Not-S,  it  will  tend  to  make  B  be  Not-S,  which 
will  tend  to  make  C  take  on  the  state  S,  which  will  make  A  take  on  the  state  S,  and  so  on.  It  is 
obvious  that  It  Is  impossible  for  the  final  state  of  the  network  to  match  the  constraints  perfectly,  and 
this  leads  to  a  harmony  score  below  1.  Also,  when  Initial  activations  are  1  for  all  units,  the  network 
takes  a  relatively  long  time  (ten  cycles)  to  settle  to  the  relatively  unharmonious  solution  it  arrives  at. 

TRe  experiential  counterpart  of  this  is  that  it  would  be  very  difficult  for  the  human  thinking'll 
process  to  come  up  with  an  entirely  satisfactory  interpretation  of  the  thought  Two  things  that  are 
different  from  each  other  can  both  be  the  same  as  something  else*.  That  the  spreading  actuation 
process  has  this  same  difficulty  bodes  well  for  its  adequacy  as  a  simulation.  In  the  simulation,  this 
difficulty  in  thinking  satisfactorily  about  that  thought  Is  reflected  quantitatively  in  low  harmony 
scores  and  slow  settling  rates.  In  the  human  mind,  there  may  be  some  cognitive  counterpart  to 
these  quantities.  — ; 

In  contrast,  consider  the  network  at  right  In  Figure  2,  corresponding  to  the  thought: 
Things  that  are  the  same  as  each  other  (B  and  C)  can  both  b<2i  different  from  something  else  (A).* 
Expeiientially,  this  is  relatively  easy  to  think  about,  and  In  the  simulation  the  spreading  activation 
process  Is  quickly  able  to  settle  to  a  state  that  satisfies  the  constraints  perfectly.  That  Is,  if  A  Is  in 
state  S,  then  B  is  in  Not-S,  as  Is  C,  and  A  Is  in  state  S,  and  all  these  are  entirely  consistent  with  the 
constraints,  leading  to  a  harmony  score  of  +1.0,  and  taking  only  2  cycles  to  settle. 

Figure  2 

Three  Node  Networks 
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Th«  Prasant  Studiaa 

Our  hypothesis  is  that  experts  In  a  subject  matter  area,  like  history,  will  have  arrived  at  an 
internally  consistent  mental  representation  of  historical  events  in  their  area.  Therefore  the 
constraints  in  their  network  of  associations  will  be  easily  satisfied,  leading  to  high  harmony  scores 
and  fest  settling.  SImilarty,  experts  in  the  skill  of  literary  Interpretation  will  be  well-practiced  in 
arriving  at  internally  consistent  mental  representations  of  literary  texts.  Therefore  the  constraints  In 
their  network  of  association  wiil  be  easily  satisfied,  leading  to  high  harmony  scores  and  fast 
settling. 

In  contrast,  novices  in  history  and  literary  criticism  wiil  be  less  likely  to  come  up  with 
Internally  consistent  representations  of  texts  in  those  subject  matter  areas,  and  so  the  constraints 
in  their  network  of  associations  will  be  less  likely  to  be  easily  satisfied,  leading  to  lower  harmony 
scores,  and  slower  settling  for  novices  than  for  experts. 

But  even  if  novices  don't  have  prior  knowledge  representations  in  an  area,  there  are 
several  ways  in  which  they  can  construct  Internally  consistent  mental  representations  in  a  subject 
matter  area.  One  way  is  by  reading  a  coherent  text  that  provides  an  internally  consistent  textual 
representation  of  the  subject  matter  area.  If  that  textual  representation  is  learned,  the  result  can  be 
that  the  novice  has  acquired  an  Internally  consistent  representation  of  the  subject  matter  area, 
leading  to  relatively  higher  harmony  scores  and  faster  settling  rates  for  those  novices  who  have 
read  coherent  texts.  But  if  the  text  Is  less  coherent,  the  chances  of  the  novice  learning  an  internally 
consistent  representation  are  tower,  leading  to  relatively  lower  harmony  scores  and  slower  settling 
ratea.  —  _ 

Another  way  in  which  novices  can  arrive  at  an  Internally  consistent  mental  representation 
of  a  subject  matter  area,  even  if  they  are  reading  an  incoiierent  text,  is  by  using  their  reading 
abilities  and  domain  prior  knowledge  to  supplement  what  is  provided  In  the  text,  so  an  to  produce 
an  Internally  consistent  representation.  So  novices  who  are  good  readers  and/or  have  lots  of  prior 
knowledge  should  have  relatively  higher  harmony  scores  and  faster  settling  rates. 

In  the  experiments  reported  below,  we  tested  these  hypothesis  by  gathering  mental 
representations  from  experts  and  novices  who  had  read  coherent  or  Incoherent  history  or  literary 
texts,  and  then  simulated  their  thinking  process  by  spreading  activation  through  our  copy  of  their 
mental  representations,  measuring  harmony  and  settling  rate.  We  expected  higher  harmony  and 
faster  settling  rates  for  experts,  for  novices  who  read  coherent  texts,  and  for  novices  who  were 
good  readers. 


Method 


Experiment  1 


Subjects.  Eight  subject  matter  experts  were  tested.  For  the  comparison  of  the  Original  to 
the  Revised  texts,  the  novices  were  357  Air  Force  recruits,  179  of  whom  read  the  Original  text,  and 
178  the  Principled  Revision,  which  had  been  revised  to  be  more  coherent.  For  the  study  of  novices 
who  were  go<^  readers,  the  subjects  were  21 1  Air  Force  recruits  who  had  read  the  Original  text. 

Materials.  The  Original  text  was  a  1000  word  self-contained  section  from  an  Air  Force 
ROTO  training  document.  The  Revised  version  had  been  modified  to  Improve  its  coherence 
(Britton  &  Quigoz,  1991). 

Tests.  The  12  most  Important  terms  from  the  text  were  selected  by  consensus  between 
the  author  and  a  graduate  student.  All  possible  pairs  of  th^  12  terms  were  constructed  and 
presented  to  all  subjects  for  rating  on  a  seven  point  scale  labeled  from  'Very  closely  related*  to 
'Very  distantly  related.'  For  purposes  of  the  associative  network  used  In  our  analysis,  responses  of 
'Very  closely  related*  were  Interpreted  as  excitatory  links  of  +1  and  Very  distantly  related*  as 
Inhibitory  links  of  -1;  the  intermediate  values  were  interpreted  as  +.67,  +.33,  0,  -.33,  and  -.67, 
respectively. 

Analytes.  The  ratings  of  each  subject  were  entered  into  a  network  in  a  program  based  on 
the  Construction-Integration  model  of  Klntsch  (1988).  Initial  activation  values  were  set  to  1.0  for 
the  data  reported  here.  (Additional  simulations  with  Initial  activations  differing  non-systematicaily 
gave  the  same  pattern  of  results.)  Harmony  was  calculated  by  taking  the  sum  of  the  product  of  the 
correlations  between  the  final  activation  of  the  units  and  the  initial  weight  matrix,  and  scaling  it  by 


I 
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the  sum  of  the  absolute  values  of  the  weights.  The  number  of  cycles  to  settle  was  recorded  by  the 
program. 

ResuKs 

Columns  1  -3  of  Table  1  show  the  harmony  and  settling  rates  for  the  average  expert  and  for 
the  average  novice  In  the  groups  who  read  the  original  text  and  the  text  revised  to  be  more 
coherent.  The  harmony  was  significantly  higher  and  the  settling  rates  significantly  lower  for  the 
experts  than  for  the  novices  in  either  group  ft(184)  =  6.72  and  7.48,  both  jj  <  .001  for  Original  text 
harmony  and  settling  time,  and  t(l85)  ^  5.66  and  8.07  for  the  Revised  text. 

Table  1 

Harmony  and  Settling  Rates  in  Experiment  1 


Novices 
who  read 
Original 

Novices 
who  read 
Revised 

Novices 

With 

Expert 

Represen¬ 

Remain¬ 

ing 

Experts 

Text 

Text 

tation 

Novices 

(n=8) 

(n=179) 

(n"178) 

(n=68) 

(n=143) 

Harmony 

Settling 

.789* 

.286 

.360* 

.469* 

.262 

Rate 
*jj  <  .05 
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Also,  the  average  harmony  was  significantiy  higher  for  the  novices  who  read  the  revised 
text  than  those  who  read  the  original  text,  J(353)  =  3.01,  j}  <  .01  and  the  settling  rate  was  lower, 
although  the  latter  was  not  significant. 

A  separate  group  of  21 1  novices  who  read  only  the  original  text  were  factor  analyzed,  and 
three  factors  were  identified  (as  in  Britton  &  Tidw^i,  1990).  The  mental  representation  of  novices  In 
one  of  these  factors  was  significantly  correlated  with  the  representation  of  the  average  expert  (as  in 
Britton  &  Tidwell,  1990).  These  novices  were  presumably  the  more  successful  readers.  Columns 
4-5  of  Table  1  show  the  average  harmony  and  settling  rate  for  subjects  in  that  fector,  and  for  the 
remaining  subjects.  The  novices  with  the  expert  representation  had  significantly  higher  harmony 
and  faster  settling  rate,  consistent  with  the  hypothesis  that  they  had  more  internally  consistent 
mental  structures  than  the  remaining  subjects,  t(209)  >  6.37,  <  .001,  J  (209)  »  2.13,  j}  <  .05. 

Discussion 

These  results  are  consistent  with  our  expectations  that  experts,  readers  of  coherent  text, 
and  good  readers  would  have  mental  representations  that  were  more  Internally  consistent  than 
novices,  readers  of  less  coherent  texts  and  poor  readers.  We  conclude  that  the  quality  of  the 
testee's  prior  knowledge  may  be  measurable  by  calculating  harmony  and  settling  rates.  The 
remaining  studies  replicated  this  finding  with  a  different  type  of  expertise,  and  also  with  different 
materials,  experts,  and  novices. 

Experiment  2 

Subjects.  Ten  Dutch  literature  professors  were  used  as  experts,  along  with  14 
undergraduate  students  from  the  same  universities.  Rolf  Zwaan  conducted  the  experiment. 

Texts.  Two  literary  texts  were  selected  from  an  international  anthology  of  prose  poems; 
both  were  by  well-known  authors,  one  by  a  winner  of  the  Ngbel  Prize  for  literature.  One  text 
described  an  elm  log  which  had  been  sawn  a  year  before,  yet  had  produced  a  fresh  green  shoot; 
the  other  text  described  the  struggles  of  a  fly  who  had  become  stuck  In  flypaper.  The  Dutch 
translations  were  used. 

Test.  Twelve  terms  were  selected,  Induding  six  terms  representing  perennial  literary 
themes  -  love,  death,  life,  hate,  happiness  and  sadness  (these  terms  were  the  same  for  both 
passages)  -  and  six  terms  from  each  passage;  for  the  elm  log  passage  these  were  log,  green, 
branch,  saw,  tree,  chop;  for  the  fly  passage  they  were  flypaper,  old  age,  attempt,  give  up,  helpless, 
man. 

All  possible  pairs  were  constructed  and  presented  to  each  subject  after  s/he  read  the  text, 
and  each  subject  rated  them  on  the  same  seven-point  scale  as  used  In  Experiment  1. 


Results 

Table  2  shows  the  results.  The  experts  had  higher  harmony  and  faster  settling  rate  than 
the  novices,  consistent  with  our  hypotheses,  but  the  effect  was  significant  only  for  settling  rate,  t 
(22)  =  1.70,i?  <  .05. 

Table  2 

Harmony  and  Settling  Rates  In  Experiment  2 


Experts  Novices 

(n=10)  (n-14) 

Harmony  .272  .256 

Settling  Rate  9.55*  15.50 


Experiment  3 

A  well-written  literary  text  can  be  given  a  coherent  literary  interpretation.  A  well  written 
informational  text  can  be  given  a  coherent  Infomtational  interpretation.  But  suppose  a  reader 
attempts  to  give  a  literary  text  an  informatlonsU  interpretation:  Attempting  to  interpret  a  text 
differently  from  the  way  its  author  Intended  will  presumably  be  more  difficult,  and  the  resulting 
mental  representation  will  likely  be  less  internally  consistent.  If  so,  harmony  will  be  lower  when 
readers  are  asked  to  interpret  a  literary  text  as  a  literal  one,  and  settling  rate  will  be  slower.  In 
Experiment  3  we  asked  novices  to  provide  literal  Interpretations  for  four  literary  texts;  novices  In  the 
control  condition  were  asked  to  produce  a  literary  Interpretation  for  the  same  four  texts. 

Subjects.  Seventy  undergraduates  were  tested,  half  in  the  Literary  Condition  and  half  In 
the  Literal  Condition. 

Texts.  The  Elm  Log  and  Ry  texts  were  used  along  with  two  additional  literary  texts  from 
the  same  source,  both  by  Nobel  Prize  winners  in  literature,  one  about  a  dream  of  a  chess  game, 
and  the  other  about  ants  returning  to  a  burning  log.  For  each  text,  a  literary  title  was  provided  for 
the  literary  group  and  a  literal  title  for  the  literal  group.  The  literary  title  was  constructed  to  cue  a 
litetary  interpretation:  for  the  Elm  Log  passage  It  was  ’Rebirth  of  the  Spirit* ;  for  the  Rypaper 
passage  it  was  ‘How  Man  Ages  and  Dies’.  The  corresponding  literal  titles  were  ’Description  of  an 
Elm  Log’  and  *A  Description  of  how  a  Ry  becomes  Tangled  in  Rypaper*.  Titles  were  constructed 
similarly  for  the  other  two  passages. 

Tests.  The  same  six  perennial  theme  terms  were  used,  along  with  six  terms  for  each 
passage,  tested  as  in  Experiments  1  and  2.  For  each  text,  about  half  of  the  terms  were  more 
susceptible  to  a  literal  interpretation  and  half  to  a  more  literary  Interpretation.  (For  some  passages, 
we  judged  that  one  or  more  perennial  theme  terms  could  more  easily  be  interpreted  literally  in  the 
context  of  that  passage,  so  the  same  number  of  other  terms  were  selected  to  be  more  susceptible 
to  a  literary  Interpretation.) 

ResuHs  and  Discussion 

Table  3  shows  the  results.  The  novices  who  provided  the  literary  Interpretation  had  higher 
harmony  and  faster  settling  rate  than  those  who  provided  the  literal  Interpretation,  consistent  with 
our  hypotheses,  but  only  the  harmony  difference  was  statistically  reliable,}  (138)  -  2.42,  ij  <  .02. 

Table  3 

Harmony  and  Settling  Rates  in  Expedment  3 

Literary  Versions  Literal  Versions 

Harmony  .170*  .114 

Settling  Rate  14.99  15.75 

Experiment  4 

This  experiment  examined  the  ability  of  experts  to  provide  a  literal  interpretation  of  a 
literary  texL 


Subjects.  The  experts  were  9  faculty  members  professionally  concerned  with  literary 
Interpretation,  from  the  Departments  of  English,  Comparative  Literature,  Romance  Languages,  and 
Philosophy  at  the  University  of  Georgia. 

Materials.  The  texts  and  tests  were  the  same  as  in  Experiment  3. 

Procedure.  Each  expert  provided  both  a  literary  and  a  literal  interpretation  for  each  text, 
on  two  occasions  separated  by  at  least  one  month. 

Results 

Table  4  shows  the  results.  The  experts  had  higher  harmony  and  faster  settling  rates  when 
they  provided  the  literary  interpretations  than  the  literal  ones,  but  the  effect  was  significant  only  for 
harmony  scores  i(8)  »  267,  £  <  .os.  That  the  average  harmony  for  the  experts'  interpretations  of 
the  literal  versions  is  negative  is  particuiarty  notable.  It  appears  that  literary  experts  have 
exceptional  difficulty  providing  a  literal  Interpretation  for  a  literary  text. 

Table  4 

Harmony  and  Settling  Rates  in  Experiment  4 

Literary  Versions  Literal  Versions 

Harmony  .230*  -.050 

Settling  Rate  12.00  20.00 


General  Discussion 

These  results  show  that  the  quality  of  a  person's  mental  representation  can  be  measured 
by  spreading  activation  through  it,  and  observing  its  settling  rate  and  harmony.  Validation  of  this  \ 

^  was  obtained  by  comparing  (a)  mental  representations  whose  quality  was  known  to  beliigh  .a 
;  priori.  l.e.,  those  of  experts,  those  of  novices  who  arrived  at  representation  similar  to  experts,  and  | 

i  both  experts  and  novices  who  read  texts  that  permitted  a  coherent  interpretation;  to  (b)  mental  | 

I  representations  which  were  known  to  be  of  lower  quality^  priori:  novices,  especially  novices  who  ! 

;  did  not  arrive  at  representations  like  those  of  experts,  and  both  experts  and  novices  who  read  texts  j 

I  less  susceptible  to  a  coherent  interpretation.  In  all  cases  the  results  were  consistent  with  j 

\  expectations.  J 

U  1  he  validity  of  this  measure  suggests  it  as  a  solution  to  a  long-standing  problem  in  the 

theory  of  metacognition;  How  does  the  person  come  to  realize  that  there  is  something  wrong  with 
his  mental  representation,  wrong  enough  to  trip  off  fix-up  mental  activities,  such  as  rereading  the 
text  to  try  to  understand  it  better,  asking  questions,  etc.?  One  way  to  know  that  you  dont 
understarid  Is  to  compare  your  understanding  with  the  correct  understanding,  but  this  is  not 
possible  unless  you  already  have  the  con'ect  understanding,  in  contrast,  it  is  always  possible  to 
calculate  harmony,  and  the  self-calculation  of  harmony  can  provide  a  feedback  signal,  which  may 
correspond  experientially  to  the  feeling  that  your  understanding  is  Incomplete.  This  signal  could 
then  be  used  to  trip  off  mental  activities  that  will  induce  fix-up  actions. 

Hamiony  also  provides  a  measure  of  quality  that  is  independent  of  comparisons  to 
experts,  which  may  prove  useful  when  experts  have  differing  views,  as  often  happens  in  complex 
subject  matter  areas,  or  when  qualified  expects  are  not  available.  For  example,  in  evaluating  the 
representation  of  a  person  who  is  the  only  expert  in  a  field,  or  in  which  there  are  no  experts, 
harmony  may  be  the  only  measure  available. 
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